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Abstract Body 


Background / Context: 

In light of the evidence that high quality early childhood education programs can improve 
children’s school readiness and future academic success (Duncan & Magnuson, 2013; Gormley, 
2008; Magnuson, Ruhm, & Waldfogel, 2007), there are a number of recent efforts at the federal 
and state levels to expand public preschool programs. These expansions aim to serve not 
just more children, but to also serve younger children (e.g. President Obama’s “Preschool for 
Ah” initiative). Federal Head Start (HS) and state prekindergarten programs (pre-k) are the two 
most prominent early education policy programs and serve millions of children nationwide. The 
individual effectiveness of each program in preparing young low-income children for school — in 
addition to their relative effectiveness — is under constant scrutiny. 

Evidence suggests that children who participate in pre-k programs have improved early learning 
outcomes in kindergarten (Gormley & Gayer, 2005; Henry, Gordon, & Rickman, 2006; Wong, 
Cook, Barnett, & Jung, 2008). Research also shows improvements in early learning skills as a 
result of HS participation (Gormley, Phillips, Adelstein, & Shaw, 2010; Ludwig & Phillips, 

2008; Puma, Bell, Cook, & Heid, 2010). However, few studies directly compare the 
effectiveness of these two programs with respect to children’s readiness for school. Henry and 
colleagues compare pre-k and HS participants using propensity score matching and find that 
economically disadvantaged children who participated in state pre-k had slightly higher scores at 
kindergarten entry relative to similar children who attended HS (2006). Gormley and colleagues 
do not match children between the two programs, but do find larger program effect sizes for 
children who attended state pre-k compared to low- income children who attended HS (2010). 

Another important policy issue is the dosage of early learning programs. There is little empirical 
work to date on this topic, but researchers have started to attend to issues of dosage in early 
learning. While some preschool programs produce large effects in only one year (Gormley, 
2008), other early learning interventions that target low-income children like Abecedarian and 
Perry Preschool provided two to five years of program services (Campbell, Pungeho, Miher- 
Johnson, Burchinal, & Ramey, 2001; Schweinhart, 2005). The HS Impact study also found 
stronger program effects for children who participated in the program at age three relative to 
those who participated at age four (Puma et ah, 2010). The body of evidence suggests that more 
participation in center-based early learning programs is associated with stronger cognitive 
outcomes, especially for low-income children (Bearing, McCartney, & Taylor, 2009; Loeb, 
Fuller, Kagan, & Carrol, 2004; NICHD Early Child Care Research Network, 2000). Still, the 
research on dosage has not rigorously controlled for selection into treatments and dosage. If 
longer exposure produces greater outcomes, this would justify increasing access for earlier entry 
into public programs. However, if the single year approach — that which is most common across 
pre-k and HS programs — produces similar outcomes, then limited public resources could be 
directed towards other child development policies. 

In our study, we replicate and innovate upon some of the analyses conducted by Gormley and 
colleagues (2005, 2008, 2010) in their evaluations of the Oklahoma state pre-k and HS programs 
in Tulsa. We use propensity score matching in addition to regression discontinuity to make 
direct statistical comparisons of the treatment effects of these two major programs on children’s 
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readiness for sehool. Furthermore, we examine whether the dosage of public early learning 
programs affects children’s pre-academic skills at kindergarten entry by creating treatment 
groups based on children’s age three and age four early learning experiences, which we refer to 
as preschool ‘pathways’, and make direct comparisons between these exposures. 

Purpose / Objective / Research Question / Focus of Study: 

The objectives of the study are to compare the overall effects of a high-quality state pre-k 
program and Head Start, compare the effects of different pathways into pre-k and Head Start, 
and to test whether two years of participation in public early learning programs (age three and 
age four participation) are more effective than one year (age four participation only). 

Research questions: 

1 . Are there differences in the effects of a high-quality pre-k program relative to a Head 
Start program on children’s pre-academic skills at school entry? 

2. Do the effects of a high-quality pre-k program at age four vary by children’s exposure to 
Head Start at age three to affect children’s pre-academic skills at school entry? 

3. Do the effects of Head Start at age four vary by children’s exposure to Head Start at age- 
three to affect children’s pre-academic skills at school entry? 

4. Does participation in Head Start at age three have a different effect on children’s pre- 
academic skills at school entry depending on the type of preschool exposure at age-fourl 

Setting and Intervention: 

This study is a secondary data analysis of data collected on preschool and kindergarten-aged 
children in Tulsa, Oklahoma in the fall of 2006 to study the effects of school and community- 
based public early learning programs on children’s cognitive and social-emotional outcomes. 
Children in the study participated in the state-funded pre-k program, in the county HS program, 
or a combination of both. The study treatment groups are described in further detail below. 

Population / Participants / Subjects and Data Collection: 

The Tulsa 2006-07 Public Use Data File contains data collected on the Oklahoma’s state-funded 
universal pre-k program administered in Tulsa Public Schools, and the Tulsa County Head Start 
program administered by local Community Action Project sites. The data include students who 
were entering: the state pre-K program, the Head Start four-year-old program, or public school 
kindergarten in the 2006-07 school year. In the prior OK pre-k and HS studies, the children who 
are entering the two preschool treatments (pre-k or HS) serve as the comparison group for the 
children who are entering kindergarten and already completed one of the two preschool 
treatments. The dataset is structured to facilitate this comparison. Within the cohort of children 
entering kindergarten, there are children who participated in either pre-k. Head Start, or in 
neither of these public programs in the year prior to kindergarten (age four). The data also 
include information on whether the child participated in HS at age three. The overall sample 
includes 1,985 pre-k entrants, 739 HS entrants, and 4,121 kindergarten entrants (1597 pre-k 
graduates, 474 HS graduates, 2050 who did not attend a public preschool program). 

Our study redefines the sample by considering children’s public preschool exposures at both age 
four and age three, creating four groups. These four pathways and their sample sizes are: 
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participants in pre-k at age four who did not participate in HS age three (3,260 total; eomposed 
of 1754 kindergarten entrants and 1503 pre-k entrants), partieipants in pre-k at age four who 
partieipated in HS at age three (322 total; 192 kindergarten entrants and 130 pre-k entrants), 
partieipants in HS at age four who did not partieipate in HS at age three (733 total; 435 
kindergarten entrants, 295 HS entrants), and participants in HS at age four and age three (480 
total; 300 kindergarten entrants, 179 HS entrants). Child and family charaeteristies by presehool 
pathway are presented in Table 1. 

The data eome from four sources: direet eognitive assessments of ehildren at the beginning of the 
sehool year; parent surveys eolleeted at their ehild’s eognitive assessment; soeial-emotional 
assessments eondueted by eaeh ehild’s teaeher; and administrative data from Tulsa Publie 
Sehools and Head Start. The three aeademie subtests used were: the Letter-Word Identifieation 
test (pre-reading skills); the Spelling test (pre-writing skills); and the Applied Problems test (pre- 
math skills) (Woodeoek & Johnson, 1989). The same subtests of a eomparable Spanish test, the 
Woodeoek-Munoz Bateria, were given to Hispanie students eapable of being tested in Spanish. 

Research Design and Data Analysis: 

The information presented in Table 1 illustrates the differenees in observed eharaeteristics 
between eaeh of the study treatment groups (presehool pathways). We use propensity score 
matching (PSM) to adjust for observable differenees between ehildren in each of the preschool 
pathways and create balanee in eovariates aeross groups where there is evidenee of seleetion into 
treatment. PSM ean mitigate the bias in treatment effeet estimates due to this seleetion on 
observed variables, assuming that there is no confounding due to unobserved variables 
(Heekman, lehimura, & Todd, 1998; Rosenbaum & Rubin, 1983). PSM was implemented using 
nearest-neighbor matehing with replaeement and a 0.01 ealiper, and we used z-tests, t-tests, and 
standardized mean differenees of oharaeteristies between samples to assess balanee (Caliendo & 
Kopeinig, 2008). 

We outlined four treatment eontrasts of interest to test our three researeh questions: 1) age four 
pre-k vs. age four HS (ignoring any HS treatment at age three), 2) age four pre-k only vs. age 
four pre-k and age three HS, 3) age four HS only vs. age three and age four HS, 4) age four pre-k 
and age three HS vs. age four HS and age three HS. We created propensity scores by running a 
logit model predieting one of the two exposures in each contrast, pulling matches from the other 
exposure, and then estimated an RD model with the matehed sample. 

Using our propensity-seore matehed groups, we then implement a regression discontinuity (RD) 
design to estimate program effeets on outeomes at kindergarten entry, and refer to these models 
as RDPSM. The primary eondition for an RD analysis is the use of a quantitative assignment 
variable and a designated eutoff score to determine treatment status (Imbens & Lemieux, 2008; 
Shadish, Cook, & Campbell, 2002). This method exploits the faet that the presehool programs 
implemented a striet age eutoff for partieipation based on ehild’s birth date, so that ehildren who 
were born before the cutoff (September 1 of 2005-06 sehool year) were eligible to partieipate in 
the age-four programs, and those bom after the eutoff were not. Thus, ehild age (measured as 
distanee between their birthdate and the eutoff birthdate in days) ean be used as the assignment 
variable in an RD speeifieation. We inelude an interaetion term between the treatment dummy 
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(born before the eutoff^l) and an indieator for one of the two presehool pathways of interest 
(e.g. treatment*age three and age four HS) to test for differential effeets between the two 
exposures being eontrasted in eaeh analysis. We used a modest bandwidth restrietion of 270 days 
(3/4 year) to ensure exehangeability in observations on either side of the treatment eutoff while 
also preserving power and preeision in smaller treatment groups (Sehoehet et ah, 2010). 

Outeome models also eontrol for parent’s edueation, ehild raee, sex, free or redueed-priee luneh 
status, exposure to other non-parental eare prior to presehool (yes=l), and missing data 
indieators. 

Findings / Results: 

The RDPSM results are displayed for eaeh researeh question in Tables 2-5, with the standard RD 
for the two eomparison groups without PSM presented in the eolumn next to the RDPSM 
estimates for eomparison. Coeffieients represent ehanges in raw seale seores. Findings indieate 
that ehildren who partieipated in pre-k have stronger reading and language skills than ehildren 
who partieipated in HS, but both programs improved ehildren’s skills. Children who partieipated 
in age three HS in addition to an age four presehool program do not have signifieantly different 
pre-aeademie skills than ehildren who partieipated in a program at age four only. OK pre-k has a 
eonsistently strong positive effeet on ehildren’s pre-reading, pre-math, and pre-writing skills at 
the beginning of kindergarten. Results indieate that in many eases the RDPSM makes a 
differenee in the magnitude and signifieanee for some eomparisons and for some outeomes, with 
ehanges in both direetions (i.e. signifieant to not, inerease in magnitude, deerease in magnitude) 
when eompared to a regular RD that ineludes that does not use PSM. 

Conclusions: 

While both pre-k and HS are effeetive at improving ehildren’s pre-aeademie skills at sehool 
entry after one year of partieipation, the high-quality OK pre-k program produees stronger gains 
in skills at the start of kindergarten than the OK HS program. We find that an additional year of 
HS at age three does not have a diseemible value-added for either age four pre-k or age four HS 
partieipation. This is in eontrast to findings by Burehinal and eolleagues (under review) that 
ehildren who entered HS at age three and also partieipated at age four had modestly higher 
voeabulary seores relative to ehildren who partieipated in HS at age four only, and other studies 
that find dosage effeets of high-quality early learning programs. 

Study limitations inelude: 1) the OK pre-k program is not representative of most state pre-k 
programs beeause of its very high quality standards; 2) ehildren living in Tulsa, OK are not 
representative of the broader population of ehildren in the U.S.; 3) propensity seore matehing 
methods assume there is no unobserved eonfounding, whieh not testable; 4) sample sizes for 
some of the treatment pathways may not provide suffieient power to deteet effeets, and the age 
four pre-k only (no age three HS) sample was signifieantly larger than the other three pathways, 
and; 5) HS and pre-k have different goals and may often serve different populations. While HS 
supports ehild eognitive, emotional, and physieal development for very low ineome ehildren, 
pre-k programs foeus solely on aeademie aetivities to prepare ehildren for sehool entry, and also 
may be offered to any ehild who is age-eligible regardless of ineome or need. 
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Table 1: Descriptive statistics 

(1) 

Age 4 pre-k; 
nothing age 

3 

Variables 

by preschool 

(2) 

Age 4 pre-k; 
Age 3 HS 

pathway 

( 3 ) 

Age 4 HS; 
nothing Age 
3 

( 4 ) 

Age4 HS; 
Age 3 HS 



Mean or proportion 


Reduced-price lunch 

0.20 

0.10 

0.06 

0.03 

Black 

0.32 

0.62 

0.21 

0.44 

Hispanic 

0.26 

0.19 

0.59 

0.40 

Asian/Native/Other 

0.11 

0.09 

0.06 

0.07 

Below HS 

0.23 

0.10 

0.33 

0.22 

Some college 

0.36 

0.38 

0.24 

0.36 

College + 

0.04 

0.10 

0.03 

0.06 

Child had some non-parental care at age 3 

0.54 

0.65 

0.44 

0.53 

Internet in the home 

0.37 

0.39 

0.25 

0.34 

Number of books in the home (scale) 

2.42 

2.21 

1.82 

2.22 

Parent is foreign born 

0.21 

0.15 

0.50 

0.35 

English is home language 

0.79 

0.84 

0.52 

0.68 

Child has health insurance 

0.87 

0.93 

0.77 

0.89 

Parent respondent is married 

0.48 

0.31 

0.53 

0.41 

Child assessed in English and Spanish 

0.19 

0.13 

0.47 

0.33 

Father in home 

0.52 

0.41 

0.57 

0.51 

Relative age within cohort 

188.60 

186.60 

187.80 

190.40 

Female 

0.48 

0.51 

0.49 

0.54 

Health Status of child (scale) 

1.50 

1.54 

1.58 

1.53 

No medical visit in past year 

0.11 

0.08 

0.12 

0.06 

Letter-word score 

6.45 

6.73 

4.77 

5.92 

Applied math score 

10.52 

9.94 

8.65 

9.71 

Spelling score 

6.61 

6.22 

5.76 

6.28 

Observations 

1,521 

178 

373 

286 
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Table 2. RDPSM and RD results comparing age four HS vs. age four pre-k 

Gormley et al. comparisons; ignoring age three HS exposure 


Letter-word Applied Problems Spelling 


B/SE B/SE B/SE 


Pre-k effect (cutoff=1) 

RD-PSM 

3.31 

RD 

3.86 

RDPSM 

0.92 

RD 

1.47 

RDPSM 

1.95 

RD 

2.27 

0.63* 

0.35* 

0.64 

0.39* 

0.44* 

0.26* 

HS effect (HS * cutoff) 

-2.44 

-2.58 

0.15 

-0.61 

-1.29 

-1.42 

0.58* 

0.45* 

0.59 

0.49 

0.43* 

0.32* 

Observations 

876 

2140 

876 

2134 

876 

2084 

Difference 

2.44 


0.15 


0.80 


p-value of difference 

0.00 


0.80 


0.00 


Effect size of difference 

0.75 


0.03 


0.31 



Significant effects and differences are emboidened. Reference group for effect of exposure is age four pre-k. Outcome variabie is a 
raw score. Aii modeis use ciustered SEs by teacher. 


Table 3. RDPSM and RD results comparing pre-k at age four and no HS at age three vs. pre-k at age 

four and HS at age three 


Letter-word Applied Problems Spelling 



B/SE 


B/SE 


B/SE 



RD-PSM 

RD 

RDPSM 

RD 

RDPSM 

RD 


4.17 

3.94 

4.46 

1.50 

3.62 

2.40 

Age four pre-k effect (cutoff=1 ) 

1.16* 

0.35* 

1.30* 

0.40* 

0.87* 

0.27* 

Age four pre-k & age 3 HS effect 

0.04 

0.11 

-2.34 

-1.07 

-0.14 

0.13 

(exposure*cutoff) 

1.11 

0.58 

1.16* 

0.76 

0.73 

0.42 

Observations 

239 

1628 

239 

1625 

239 

1581 

Difference 

0.04 


2.34 


0.14 


p-value of difference 

0.97 


0.05 


0.85 


Effect size of difference 

0.01 


0.85 


0.05 



Significant differences are emboidened. Reference group for effect of exposure is age four pre-k, no age three HS. Outcome variabie 
is a raw score. Aii modeis use ciustered SEs by teacher. 
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Table 4. RDPSM and RD results comparing HS at age four and no HS at age three vs. HS at age four 

and HS at age three 

Letter-word Applied Problems Spelling 

B/SE B/SE B/SE 


Age four HS effect (cutoff=1) 

RD-PSM 

1.14 

RD 

0.79 

RDPSM 

2.13 

RD 

0.85 

RDPSM 

-0.37 

RD 

0.38 

1.11 

0.72 

1.14 

0.63 

0.60 

0.42 

Age four HS & age 3 HS effect 

0.20 

0.77 

0.26 

0.35 

0.93 

0.66 

(exposure*cutoff) 

0.63 

0.57 

1.02 

0.80 

0.49 

0.44 

Observations 

346 

690 

346 

686 

346 

673 

Difference 

0.20 


0.26 


0.92 


p-value of difference 

0.75 


0.80 


0.07 


Effect size of difference 

0.04 


0.05 


0.34 



Significant differences are emboidened. Reference group for effect of exposure is age four HS, no age three HS. Outcome variabie is 
a raw score. Aii modeis use ciustered SEs by teacher. 


Table 5. RDPSM and RD results comparing HS at age four and HS at age three vs. pre-k at age four 

and HS at age three 

Letter-word Applied Problems Spelling 

B/SE B/SE B/SE 



RD-PSM 

RD 

RDPSM 

RD 

RDPSM 

RD 

Age four and age three HS effect (cutoff=1) 

0.91 

1.83 

1.29 

1.43 

1.92 

1.40 

0.94 

0.81* 

1.22 

0.94 

0.89* 

0.57* 

Age four pre-k & age 3 HS effect 

2.99 

2.23 

-0.52 

-0.79 

0.58 

0.91 

(exposure*cutoff) 

0.92* 

0.78* 

1.18 

1.10 

0.76 

0.56 

Observations 

286 

466 

286 

462 

286 

446 

Difference 

2.99 


0.51 


0.58 


p-value of difference 

0.01 


0.66 


0.45 


Effect size of difference 

0.91 


0.11 


0.21 



Significant differences are emboidened. Reference group for effect of exposure is age four HS and age three HS. Outcome variabie is 
a raw score. Aii modeis use ciustered SEs by teacher. 
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